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ABSTRACT 



This study examined new admission policies at the University 
of Virginia School of Medicine designed to increase the number of students 
likely to pursue primary care specialities. The study focused on whether 
there was a relationship between interviewers' predictions and applicants' 
career choice at the completion of their first year of medical school. 
Interviewer predictions during admissions interviews were compared to the 
career choices of two cohorts of applicants who matriculated in 1994 and 
1995. For the 1994 cohort, 148 out of 232 interviewer predictions (64 
percent) were correct, while for the 1995 cohort, 161 out of 254 predictions 
(63 percent) were correct. Correlational analysis indicated a low to moderate 
relationship between the interviewers' predictions and matriculants' career 
choices. Interviewers tended to make more correct predictions regarding those 
students not likely to enter primary care. (Contains 10 references.) (MDM) 
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An Evaluation of a New Admission Policy Intended to Increase 
the Numter of Primary Care Physicians 

Background. In an effort to admit more students inclined to select primary care careers, admissions 
committee interviewers at the University of Virginia (UVa) School of Medicine predicted which 
applicants would be “most likely” to pursue primary care specialties, with the intent of utilizing these 
predictions as part of the selection process. The use of these predictions represented a change in the 
medical school admission process. The purpose of this study was to determine if there was a relationship 
between the interviewers’ predictions and the matriculants’ career choice at the completion of their first 
year of medical school. 

Method. Interviewer predictions were compared to the career choices of two cohorts of applicants 
who matriculated into the UVa School of Medicine in 1994 and 1995. Career choices for each of the two 
cohorts of matriculants were collected from a career planning survey administered at the end of the first 
academic year. The overall response rate to the survey was 95% for the 1994 matriculants and 98% for 
the 1995 matriculants. Cramer’s V Correlation Coefficient was utilized to determine relationship 
between the interviewer prediction variable and the matriculant career choice variable, and the kappa 
statistic was used to assess the level of interviewer reliability regarding career predictions. Percentages 
also were reported regarding the accurate match of individual interviewer predictions with the matriculant 
career choice. 

Results and conclusions. The correlations for the two cohorts were as follows: .307 (p = .01) for the 
matriculating class of 1994, and .345 (p = .01) for the matriculating class of 1995. The correlation of 
.307 and .345 indicated a low to moderate relationship between the interviewers’ predictions and the 
matriculants’ career choices for each of the two cohorts. The results of this preliminary study, however, 
must be viewed as a “snapshot” and not as the final comparison between the interviewers’ predictions and 
medical students’ choices of a primary care career. This final comparison can only be made when each 
student formally makes that decision many years from now. The results of this study did provide 
interviewers valuable data regarding their initial accuracy in predicting matriculants’ interest in a primary 
care career. Tliis feedback cycle enabled interviewers to make a preliminary assessment concerning their 
career predictions. 
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An Evaluation of a New Admission Policy Intended to Increase 
the Number of Primary Care Physicians 

The 1992 report by the Council on Graduate Medical Education (COGME) entitled Improving 
Access to Health Care Through Physician Workforce Reform: Directions for the 2T^ Century was 
a major document outlining our nation’s imperative need for an increase in the number of primary 
care physicians. One of the Council’s recommendations to achieve this objective was for medical 
schools to examine their admissions practices with the goal of admitting more students inclined to 
select primary care careers ' (family practice, general internal medicine, and general pediatrics). 

Tlie role and possible effects of medical schools’ admission policies on increasing the number of 
primary care physicians have been discussed by a number of writers and researchers. A number 
of studies regarding the identification of applicants most likely to enter a primary care specialty 
also have been conducted. Though the global effects of the admissions process have been 

investigated, these studies have not examined interviewers’ ability to discern between applicants 
regarding ultimate career choice. 

In an effort to admit more students inclined to select primary care careers, admissions 
committee interviewers at the University of Virginia (UVa) School of Medicine predicted ^\'hich 
applicants would be “most likely” to pursue primary care specialties, with the intent of utilizing 
these predictions as part of the selection process. The use of these predictions represented a change 
in the medical school admission process, and an evaluation of this new poUcy was vital in 
accessing whether the intended impact on primary care career selection was accomplished. The 
purpose of this study was to determine if there was a relationship between the interviewers’ 
predictions and the students’ first choice of career at the completion of their first year of medical 




school. 
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Background 

The UVa Committee on Admissions interviewed approximately 500 medical school applicants 
from September 1993 through March 1994 (1993-94 cohort), and approximately 500 applicants 
from September 1994 through March 1995 (1994-95 cohort). The Admissions Committee had an 
orientation session immediately preceding each of these two “interviewing seasons” to review 
interviewing procedures. References to pertinent literature regarding the identification of 
applicants likely to enter a primary care career were made, but interviewers were not required to 
read specific articles. 

During the “interviewing season,” each applicant had two, thirty minute interviews, each with 
an admissions committee member. During these unstructured interviews, the interviewers sought 
to evaluate the candidate’s motivation for a career in medicine, maturity, social awareness, 
independence, personal ethics, interpersonal and communication skills, commitment to lifelong 
learning, compassion, sensitivity, and tolerance. A prediction regarding the likelihood for a career 
in primary care, academic medicine, or clinical subspecialty also was made. To ensure that the 
prediction was made, the following question was included on the Interviewer Report Form: “Based 
on your review of this applicant, which field of medicine do you predict he/she is most likely to 
enter - (a) primary care/generalist medicine, (b) other clinical specialty, or (c) biomedical research 
and/or academic medicine?” These predictions were considered later when selecting those offered 
admission. 

Method 

The interview predictions for each of the 1994 matriculants (1993-94 cohort) and 1995 
matriculants (1994-95 cohort) were received in a report prepared by the admissions director. 
Interviewers were coded to ensure that they remained anonymous. 
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The career choices for each of the two cohorts of students who had completed their first year 
were collected from a survey. The survey was constructed to 5 deld data regarding the students’ top 
three career choices and their degree of certainty about each choice. A pilot test of the survey was 
conducted to enhance the validity of the instrument. The overall response was a 95% return rate 
for the 1994 matriculants, and a 98% return rate for the 1995 matriculants. 

Both the interviewer prediction variable and the student first career choice variable were 
dichotomized. Therefore, Cramer’s V Correlation Coefficient was utilized to answer the research 
question. An alpha level of .05 was set for the significance level. 

The kappa statistic was used to assess the level of interviewer reliability regarding career 
predictions. Two interviewers for each applicant, and the use of categorical data, made this a 
reasonable approach to evaluate interviewer agreement. 

Comparisons of individual interviewer’s predictions with the first choice of career for the 
students they interviewed also were reported in terms of frequencies. 

Results 

A correlation of .307 (p = .01) was obtained for the interviewers’ predictions and the 1994 
matriculants’ first choices of career, and a correlation of .345 (p = .01) was obtained for the 
interviewers’ predictions and the 1995 matriculants’ first choices of career. Both correlation 
coefficients indicated a low to moderate strength of relationship between the two variables. 

A kappa statistic of .449 was obtained for the 1993-94 cohort interviewer predictions, and .647 
was obtained for the 1994-95 cohort interviewer predictions. The .449 statistic indicated a 
moderate strength of agreement, and the .647 indicated a substantial strength of agreement, as 
interpreted by the scale suggested by Landis and Koch (Landis and Koch; 1977). 




For the 1994 matriculants, there were 232 predictions made by 26 interviewers for 116 
students. Each student had two separate interviews, and each interviewer made a prediction 
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relative to the student’s interest in a primary care career (Table 1). Deferred students who did not 
have interviewer predictions and students who did not return their surveys were excluded from the 
study. 



Table 1 

1994 Matriculants 

Interviewers’ Predictions Relative to Matriculants’ Career Choice 

Matriculants’ Choice Matriculants’ Choice 

Generalist NonGeneralist 

Interviewer Prediction 

Generalist * 76(56%) 60(44%) 

Interviewer Prediction 

NonGeneralist 24 (25%) * 72 (75%) 

• Correct predictions 

Of these 232 predictions, 148 (64%) were correct and 84 (36%) were incorrect relative to the 
first choice of career given by the 1 16 students on their surveys. Further, 136 (59%) of these 232 
predictions were “most likely” predictions that the students interviewed would enter a primary care 
career. The remaining 96 (41%) were predictions that the students interviewed would not likely 
enter a generalist career. 

Of the 136 “most likely to enter a primary care career” predictions, 76 (56%) were correct and 
60 (44%) were incorrect relative to the first choice of career given by the 1 16 students. Regarding 
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the 96 “not likely to enter primary care” predictions, 72 (75%) were correct and 24 (25%) were 
incorrect relative to the first choice of career. 

For the 1995 matriculants, there were 254 predictions made by 29 interviewers for 127 
students. Of these 254 predictions, 161 (63%) were correct and 93 (37%) were incorrect relative to 
the first choice of career given by the 127 students on their surveys (Table 2). Students who did 
not have interviewer predictions or who did not return their surveys were excluded from the study. 

Table 2 

1995 Matriculants 

Interviewers’ Predictions Relative to Matriculants’ Career Choice 

Matriculants’ Choice Matriculants’ Choice 

Generalist NonGeneralist 

Interviewer Prediction 

Generalist ♦ 82(52%) 77(48%) 

Interviewer Prediction 

NonGeneralist 16(17%) * 79(83%) 

* Correct predictions 

Examining these 254 predictions, 159 (63%) were “most likely” predictions that the students 
interviewed would enter a primary care career. The remaining 95 (37%) were predictions that the 
students interviewed would not likely enter a generalist career. 

Of the 159 “most likely to enter a primary care career” predictions, 82 (52%) were correct and 
77 (48%) were incorrect relative to the first choice of career. For the 95 “not likely to enter 
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primary care” predictions, 79 (83%) were correct and 16 (17%) were incorrect relative to the first 
choice of career given on the survey. 

Conclusion 

The purpose of this study was to determine if there was a relationship between the interviewers’ 
predictions and the students’ first choice of career at the completion of their first year of medical 
school. The correlation of .307 and .345 indicated a low to moderate relationship between the 
interviewers’ predictions and the matriculants’ career choices for each of the two cohorts. Results 
of this study, however, must be viewed as a “snapshot” and not as the final comparison between 
the interviewers’ predictions and medical students’ choices of a primary care career. 

The results of this study provided interviewers individual data regarding their initial accuracy in 
predicting matriculants’ interest in a primary care career. This feedback cycle enabled interviewers 
to make a preliminary assessment concerning their career predictions. 

It was interesting to note that the interviewers tended to make more correct predictions 
regarding those students likely not to enter a primary care career. This might indicate that 
interviewers were more astute regarding the identification of applicants more likely to enter a 
speciahst career than those likely to enter a primary care career. 

This study was part of a longitudinal, career tracking study that traces changes in medical 
students’ career choices toward primary care specialties from entry through the end of medical 
school. Data collected in this longitudinal study should provide insig)its in the career selection 
process that might be useful in training interviewers to make career predictions. 

In summary, this study provided a foundation for the longitudinal study described above, and 
can serve as a model for other schools that are implementing and evaluating new admission policies 
designed to increase the number of primary care physicians. 
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